Papers with RoBERTa language model
Prompting for Multimodal Hateful Meme Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Hateful meme classification requires complex reasoning and contextual background knowledge. |
| Approach: | They propose a simple yet effective prompt-based model that prompts pre-trained language models for hateful meme classification. |
| Outcome: | The proposed model outperforms state-of-the-art models on hateful meme classification task. |
SUPP.AI: finding evidence for supplement-drug interactions (2020.acl-demos)
Copied to clipboard
Lucy Lu Wang, Oyvind Tafjord, Arman Cohan, Sarthak Jain, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Waleed Ammar
| Challenge: | Dietary supplements are used by a large portion of the population, but information on their pharmacologic interactions is incomplete. |
| Approach: | They propose an application to search evidence sentences extracted from the literature to identify supplement-drug interactions. |
| Outcome: | The proposed model extracts supplement information and identifies interactions using labeled DDI data. |
Bernice: A Multilingual Pre-trained Encoder for Twitter (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models for Twitter are monolingual, adapted from other domains, or trained on limited amount of in-domain data. |
| Approach: | They propose a multilingual RoBERTa language model that is trained from scratch on 2.5 billion tweets with a custom tweet-focused tokenizer. |
| Outcome: | The proposed model outperforms or matches models trained on monolingual and multilingual tweets on a variety of benchmarks and is more efficient compute- and data-wise to train completely on in-domain data with a specialized domain-specific tokenizer. |